Best-Effort Request Labeling and Scheduling on Multicore Servers

Multicore servers continue to scale on the CPU count and request execution concurrency. It is desirable to support request quality-of-service such as reducing the latency distribution tail and providing differential services. This paper presents new operating system support that labels CPU executions with requests or their QoS parameters (e.g., request arrival time) and enables online request scheduling. Since OS-level request tracking suffers from inaccuracies in situations like user-space synchronization and batched event processing, we devise effective request tracking / scheduling strategies that consciously tolerate such inaccuracies. In particular, we find that ignoring partially OS-visible synchronizations causes less harm for QoS scheduling than propagating request labels through them. We also augment OS event dispatcher like epoll wait() to return a batch of events with similar (but likely unequal) scheduling labels to user space according to the QoS heuristics. We utilize our best-effort request labeling to schedule for tail latency reduction, request prioritization, and hyperthread-contention throttling. We evaluate our system using an RSA encryption workload, a CPU/cache/memory-stressing server, the Apache Solr search engine, a Google App Engine application, and an event-driven Redis hash store on a 40-CPU machine. Results demonstrate low overhead (no more than 1%) and high scheduling effectiveness (reducing the 99th percentile request response time by 8%, 24%, 35%, 50%, and 43% for the five workloads). This work was supported in part by the National Science Foundation grants CNS-1217372, CNS-1239423, and CCF-1255729, and by a Google Research Award.